Digital camera with electronic zooming function

ABSTRACT

Bayer data is captured by a CCD and is stored in a memory. Image data is read out from the memory in units of nine lines starting with (4 n +1)-th (n: integer) line. YUV data of one pixel is formed based on 5×5 pixels in the read Bayer data. Thus, the YUV data is formed in units of five lines. The YUV data is enlarged by a pixel number converting unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a Divisional Application of U.S. applicationSer. No. 10/186,530, filed Jul. 1, 2002, which is based upon and claimsthe benefit of priority from the prior Japanese Patent Application No.2001-202518, filed Jul. 3, 2001, the entire contents of which areincorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a digital camera that can enlarge animage and an image processing method for enlarging an image.

2. Description of the Related Art

Recently, the development of personal computers has contributed to thewide spread of digital still cameras that record digital image dataobtained using an image pick-up element, in a memory card in place of asilver film.

FIG. 1 shows the configuration of a circuit in a digital still camera ofthis kind. When one image is picked up (single image pick-up), theillustrated circuit basically compresses YUV data (luminance and colordifference data) before expanding one frame of the YUV data on a buffermemory.

An optical image obtained using an optical system 11 is formed as animage on a CCD 12 used as an image pick-up element. The CCD 12 includes,for example, an RGB color filter of Bayer type. Color image dataobtained (hereinafter referred to as “Bayer data”) is sequentiallysampled and held by a sample and hold circuit (S/H) 13, subsequentlydigitized by an A/D converter 14, and then supplied to a line processingunit 15.

The line processing unit 15 integrates the sequentially transmittedBayer data into line data. A first transfer unit 16 transfers the Bayerdata, integrated into the line data, to a memory 17, where the data isexpanded and stored.

Once a predetermined number of lines of Bayer data are expanded on thememory 17, these data are read in block units and transferred to aBayer/YUV converting unit 19 by a second transfer unit 18A.

The Bayer/YUV converting unit 19 executes an interpolating process or acolor space process to convert the Bayer data (RGB data) into the YUVdata, luminance and color difference color image. Then, the data thusobtained are supplied to a JPEG processing unit 21 via a switch (SW) 20.On the other hand, a third transfer unit 23 transfers these data to thememory 7 via a switch 22. Then, the data is expanded and stored in thememory 7.

The YUV data stored in the memory 17 are transferred to a displaycontrol unit 25 by a fourth transfer unit 24. Then, the display controlunit 25 generates and outputs an analog video output signal and a signalfor display on a liquid crystal monitor.

The JPEG processing unit 21 executes a data compression process such asan ADCT (Adaptive Discrete Cosine Transform) process or Huffman encodingprocess to the YUV data transferred by the Bayer/YUV converting unit 19via the switch 20. Thus, JPEG data of a drastically reduced amount isobtained. A fifth transfer unit 26 transfers the JPEG data thus obtainedto the memory 17, where the data is expanded and stored.

The JPEG data expanded and stored in the memory 17 is recorded in amemory card as a storage medium.

On the other hand, in a reproduction mode, JPEG data read from thememory card is stored in the memory 17 and then read and transferred tothe JPEG processing unit 21 by a sixth transfer unit 27. Then, the JPEGprocessing unit 21 converts the JPEG data to the original YUV data, andthe third transfer unit 23 transfers this YUV data via the switch 22 tothe memory 17 for storage. Subsequently, the data is delivered to thedisplay control unit 25 by the fourth transfer unit 24.

The above described circuit operations are totally controlled by asystem controller 28 including a CPU. The operation of the systemcontroller 28 is controlled in response to key operation signals inputfrom a key input unit 29 including a shutter key and a mode key.

Now, a specific process for generating the YUV data will be described indetail.

Bayer data obtained by the line processing unit 15 and expanded andstored in the memory 17 is illustrated in FIG. 2A.

If it is assumed that it is necessary to refer a pixel configuration of5×5 pixels of the Bayer data (FIG. 2A) as shown in FIG. 2B in order togenerate one pixel of the YUV data by the Bayer/YUV converting unit 19,the YUV data as shown in FIG. 2C is generated.

It is assumed that the YUV data generated is transferred and output inthe order shown in FIG. 3 and comprises blocks each having a verticalsize of four pixels. Then, the pixels of the original Bayer data areread in the order as shown in FIG. 4. That is, for the first block withthe range of lines C1 to C8, the pixels are read in the following order:

(C1, 1)→(C1, 2)→(C1, 3)→(C1, 4)→(C1, 5)→(C1, 6)→(C1, 7)→(C1, 8)→(C2,1)→(C2, 2)→ . . .

For the second block the starting line of which is shifted downward byfour lines and which has the range of lines C5 to C12:

(C1, 5)→(C1, 6)→(C1, 7)→(C1, 8)→(Cl, 9)→(C1, 10)→(C1, 11)→(C1, 12)→(C2,5)→(C2, 6)→ . . .

Compared to the Bayer data read in this pixel position order, the pixelsof the YUV data is transferred in the following order. For the firstblock:

(C3, 3)→(C3, 4)→(C3, 5)→(C3, 6)→(C4, 3)→(C4, 4)→(C4, 5)→(C4, 6)→(CS,3)→(CS, 4)→ . . .

For the second block:

(C3, 7)→(C3, 8)→(C3, 9)→(C3, 10)→(C4, 7)→(C4, 8)→(C4, 9)→(C4, 10)→(CS,7)→(C5, 8)→ . . .

FIG. 5A shows the configuration of pixels obtained when the YUV data isgenerated from Bayer data as described above. In the Bayer data in FIG.5A, the hatched pixel shows a pixel used in generating the YUV data butwhich does not directly generate the corresponding YUV data.

In a Bayer data block read from the memory 17 and comprising eightlines, upper four lines are also read in the upper adjacent block. Thus,the YUV data generated in correspondence with these lines is configurednot to contain any duplicate lines.

FIG. 5B illustrates the relationship between a Bayer data block readfrom a Bayer data frame and a YUV data block generated from the Bayerdata block.

As described above, when the YUV data is generated from the Bayer data,the minimum required YUV data is generated, as shown in FIG. 5A.

Accordingly, the vertical size of the YUV data block is set at 8 if therequired data is 4:2:2 (Y:Cb:Cr), and is set at 16 if the required datais 4:2:0. Then, the JPEG processing unit 21, which subsequentlycompresses the data, executes the process on every block comprising 8×8pixels. Consequently, the YUV data can be transferred directly to theJPEG processing unit 21.

Here, with the circuit configuration shown in FIG. 1, in order toincrease the speed of an electronic zooming process (enlarging process)in which the image data obtained through image pick-up is interpolatedto increase the number of pixels constituting the image, it is assumedthat a circuit that executes a pixel number conversion using hardware isprovided succeeding to the Bayer/YUV converting unit 19, for example,between the switch 22 and the third transfer unit 23, instead ofexecuting the enlarging process after expanding and storing one frame ofYUV data on the memory 17. On these assumptions, since only the minimumrequired YUV data is transferred as described above, the interpolatingprocess fails to generate pixels located between the pixels on thelowermost line of a YUV data block and the pixels on the uppermost lineof the next YUV data block.

Thus, on the assumption that the same pixels as those on the lowermostline of the YUV data block are added as a line immediately below thelowermost line or that the pixels on lowermost line of the YUV datablock are overlapped with the line immediately below the lowermost line,the pixel number converting circuit must execute an interpolatingprocess (enlarging process) using these duplicate pixels. As a result,disadvantageously, the image is more markedly degraded as theenlargement ratio for electronic zooming increases in the verticaldirection.

Further, it is assumed that the pixel number converting circuit executesan electronic zooming process (enlarging process) on each YUV datablock. Then, since each YUV data block contains the same number oflines, it contains the same number of lines even after the enlargingprocess (interpolating process). Thus, disadvantageously, the availableenlargement ratio (enlargement ratio) is limited.

Furthermore, if the pixel number converting circuit is to execute anelectronic zooming process (enlarging process) on YUV data blocks andthe YUV data block generated by the Bayer/YUV converting process istransferred directly to the JPEG processing unit 21, the YUV data, whichhas not undergone any electronic zooming process, is compressed andstored in the memory card. To avoid this, it is necessary to store theYUV data already subjected to an electronic zooming process in thememory 17 and transferring the read data to the JPEG processing unit 21.Consequently, much time is required to record the image.

BRIEF SUMMARY OF THE INVENTION

The present invention is directed to method and apparatus thatsubstantially obviates one or more of the problems due to limitationsand disadvantages of the related art.

According to an embodiment of the present invention, a digital cameracomprises:

an image pick-up unit that picks up an image of an object and outputsimage data;

a transfer unit that transfers the image data output from the imagepick-up unit in units of a predetermined number of lines, with at leastone line overlapping at least one line of an immediately following unit;and

an enlarging process unit that enlarges the image data transferred fromthe transfer unit in units of the predetermined number of lines.

Additional objects and advantages of the invention will be set forth inthe description which follows, and in part will be obvious from thedescription, or may be learned by practice of the invention. The objectsand advantages of the invention may be realized and obtained by means ofthe instrumentalities and combinations particularly pointed outhereinafter.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the presentinvention and, together with the general description given above and thedetailed description of the embodiments given below, serve to explainthe principles of the present invention in which:

FIG. 1 is a block diagram showing the configuration of a circuit in aconventional digital still camera;

FIGS. 2A, 2B, and 2C are diagrams showing the concept of generating theYUV data from the Bayer data;

FIG. 3 is a diagram showing the concept of generating the YUV data fromthe Bayer data;

FIG. 4 is a diagram showing the concept of generating the YUV data fromthe Bayer data;

FIGS. 5A and 5B are diagrams showing the concept of generating the YUVdata from the Bayer data;

FIG. 6 is a block diagram showing the configuration of a circuit in adigital still camera according to the first embodiment of the presentinvention;

FIG. 7 is a flow chart showing the contents of a process executeddepending on whether or not an electronic zooming function is to beexecuted in a pick-up mode, according to the first embodiment;

FIG. 8 is a diagram showing a relationship between a Bayer data transferstate and the YUV data correspondingly generated;

FIG. 9 is a diagram showing a different relationship between the Bayerdata transfer state and the YUV data correspondingly generated;

FIG. 10 is a block diagram showing the configuration of a circuit in adigital still camera according to the second embodiment of the presentinvention;

FIG. 11 is a flow chart showing the contents of a process executeddepending on whether or not the electronic zooming function is executedin the pick-up mode, according to the second embodiment;

FIG. 12 is a block diagram showing the configuration of a circuit in adigital still camera according to the third embodiment of the presentinvention;

FIG. 13 is a diagram showing specific correspondences between pixels inan enlarging process according to the third embodiment;

FIG. 14 is a block diagram showing the configuration of a circuit thatextracts pixels according to the third embodiment;

FIG. 15 is a diagram illustrating an output form of line number dataaccording to the third embodiment; and

FIG. 16 is a diagram illustrating another output form of line numberdata according to the third embodiment.

DETAILED DESCRIPTION OF THE INVENTION

An embodiment of a digital still camera according to the presentinvention will now be described with reference to the accompanyingdrawings.

First Embodiment

FIG. 6 is a block diagram showing the configuration of the firstembodiment. The same portions as those shown in FIG. 1 will be indicatedin the same reference numerals and their detailed description will beomitted.

As shown in FIG. 2B, it is assumed that 5×5 pixels of Bayer data arerequired to generate one pixel of YUV data and a YUV data blockgenerated from the Bayer data requires a vertical size of at least fourpixels. In this case, the second transfer unit 18 for reading the Bayerdata stored in the memory 17 reads the Bayer data in units of ninelines, while the data being shifted by four lines.

The circuit has a pixel number converting unit (enlarging process unit)31 connected between the switch 22 that opens and closes a transfer pathfor YUV data blocks sequentially generated by the Bayer/YUV convertingunit 19 and the third transfer unit 23 that expands and stores the YUVdata blocks sequentially transferred via the switch 22, in the memory17.

When an electronic zooming function is turned on, the pixel numberconverting unit 31 enlarges the sequentially input YUV data blocks,thereby increasing the number of pixels in the block according to anenlargement ratio.

Now, the operation of this embodiment will be described.

FIG. 7 shows the contents of a process executed depending on whether ornot an electronic zooming operation is to be executed in a pick-up mode.First, it is determined whether or not an electronic zooming function isturned on using a zoom key included in the key input unit 29 (step A01).

If the electronic zooming function has not been turned on, the secondtransfer unit 18 reads the Bayer data in units of one block comprisingeight lines from the memory 17 such that the upper four lines of thedata block are overlapped by lower four lines of the upper adjacent datablock, as shown in FIG. 5A. The second transfer unit 18 then transfersthis block to the Bayer/YUV converting unit 19.

Thus, YUV data blocks sequentially generated by the Bayer/YUV convertingunit 19 on the basis of the Bayer data contain no duplicate pixels. TheYUV data blocks sequentially generated pass through the pixel numberconverting unit 31 via the switch 22 to reach the third transfer unit23. The third transfer unit 23 transfers the data blocks to the memory17, where the data blocks are expanded and stored. The fourth transferunit 24 then transfers the data blocks to the display control unit 25. Aliquid crystal monitor of this digital still camera displays and outputsthe data blocks (step A03).

If it is determined at step A01 that the electronic zooming function isturned on, the second transfer unit 18 reads the Bayer data in units ofone block comprising nine lines from the memory 17 as shown in FIG. 8.The second transfer unit 18 then transfers this block to the Bayer/YUVconverting unit 19.

For example, the pixels of the Bayer data are read by the secondtransfer unit 18 in the following order. For the first block with therange of lines C1 to C9 shown in FIG. 4:

(C1, 1)→(C1, 2)→(C1, 3)→(C1, 4)→(C1, 5)→(C1, 6)→(C1, 7)→(C1, 8)→(C1,9)→(C2, 1)→(C2, 2)→ . . .

For the second block the starting line of which is shifted downward byfour lines and which has the range of lines C5 to C13:

(C1, 5)→(C1, 6)→(C1, 7)→(C1, 8)→(C1, 9)→(C1, 10)→(C1, 11)→(C1, 12)→(C1,13)→(C2, 5)→(C2, 6)→ . . .

Accordingly, the Bayer/YUV converting unit 19 generates five lines ofYUV data from nine lines of Bayer data. For example, the pixels of YUVdata generated are transferred in the following order. For the firstblock:

(C3, 3)→(C3, 4)→(C3, 5)→(C3, 6)→(C3, 7)→(C4, 3)→(C4, 4)→(C4, 5)→(C4,6)→(C4, 7)→(C5, 3)→(CS, 4)→ . . .

For the second block:

(C3, 7)→(C3, 8)→(C3, 9)→(C3, 10)→(C3, 11)→(C4, 7)→(C4, 8)→(C4, 9)→(C4,10)→(C4, 11)→(C5, 7)→(CS, 8)→ . . .

In this manner, the YUV data in the block located at a certain positionis generated so that the lowermost line of the data is overlapped with(is the same as) the uppermost line of the YUV data in the next block,as shown by the hatching in FIG. 8.

On the YUV data block generated so as to contain one duplicate line, thepixel number converting unit 31 executes, via the switch 22, a pixelconverting process including an image enlarging process corresponding tothe enlargement ratio. During this enlarging process, the YUV data blockis intentionally transferred so as to partially overlap the followingdata block as described above. Consequently, when pixels lying betweenblocks are generated, an image to be generated is not degraded.

Specifically, the enlarging process comprises referencing 2×2 pixels ofthe YUV data to generate pixel data to be located in the center of thesepixels, by interpolation. Since a duplicate line is added to the YUVdata block as the fifth line, pixel data for an interpolating linebetween the fourth line of this data block and the first line of thenext data block can be generated by using the YUV data in the fourth andfifth lines and referencing an area of 2×2 pixels in each of these lineswhen the fourth line of the YUV data is to be processed.

Then, the third transfer unit 23 transfers the YUV data blockssequentially obtained through the enlarging process and each having anincreased number of lines, to the memory 17, where the data blocks areexpanded and stored. The fourth transfer unit 24 then transfers each YUVdata block to the display control unit 25. The liquid crystal monitor ofthis digital still camera displays and outputs a required display range(step A02).

In this case, the YUV data in the fifth line of the block is only usedby the pixel number converting unit 31 to execute an enlarging processfor electronic zooming. The third transfer unit 23 does not transfer theYUV data in this fifth line to the memory 17.

The monitor display process is thus executed at step A02 or A03depending on whether or not the electronic zooming function is turnedon. Then, while repeatedly executing a process of determining whether ornot the shutter key included in the key input unit 29 is operated (stepA04) and if not, returning to the process starting with step A01, theapparatus waits for the shutter key to be operated.

If it is determined at step A04 that the shutter key is operated, thenit is determined whether or not the electronic zooming function isturned on (step A05).

If the electronic zooming function has not been turned on, a normalimage recording process is executed. That is, the Bayer data obtainedthrough an image pick-up operation performed by the CCD 12 at a nexttiming will be transmitted to the first transfer unit 16 via the sampleand hold circuit 13, A/D converter 14, and line processing unit 15. Thefirst transfer unit 16 then transfers the Bayer data to the memory 17for storage. Then, the second transfer unit 18 reads every block of theBayer data comprising eight lines from the memory 17. Subsequently, theBayer/YUV converting unit 19 generates the YUV data blocks eachcomprising four lines and having no duplicates as shown in FIGS. 5A and5B.

The YUV data blocks sequentially generated by the Bayer/YUV convertingunit 19 and each comprising four lines are sequentially transferred tothe JPEG processing unit 21 via the switch 20. Each of the data blocksis then compressed to obtain the JPEG data in units of block. The fifthtransfer unit 26 then transfers one frame of JPEG data to the memory 17for storage and then records these data in a memory card (step A07).

Once the normal image recording process is executed, the process returnsto step A01 to provide for the next pick-up session.

If it is determined at step A05 that the electronic zooming function isturned on when the shutter key is operated, then an image recordingprocess including an image enlarging process is executed. The Bayer dataobtained through an image pick-up operation performed by the CCD 12 at anext timing will be transmitted to the first transfer unit 16 via thesample and hold circuit 13, A/D converter 14, and line processing unit15. The first transfer unit 16 then transfers the Bayer data to thememory 17 for storage. Then, the second transfer unit 18 reads everyblock of the Bayer data comprising nine lines from the memory 17.Subsequently, the Bayer/YUV converting unit 19 generates the YUV datablocks each comprising five lines including one duplicate line as shownin FIG. 8.

The YUV data blocks generated by the Bayer/YUV converting unit 19 andeach comprising five lines are sequentially transferred to the pixelnumber converting unit 31 via the switch 22. The pixel number convertingunit 31 then enlarges the data block so as to increase the number ofpixels according to the currently set enlargement ratio. Subsequently,the third transfer unit 23 transfers the obtained YUV data block to thememory 17 for storage.

Specifically, this enlarging process comprises referencing 2×2 pixels ofthe YUV data to generate pixel data to be located in the center of thesepixels, by interpolation, as described above. Since a duplicate line isadded to the YUV data block as the fifth line, pixel data for aninterpolating line between the fourth line of this data block and thefirst line of the next data block can be generated by using the YUV datain the fourth and fifth lines and referencing an area of 2×2 pixels ineach of these lines when the fourth line of the YUV data is to beprocessed.

Subsequently, the sixth transfer unit 26 transfers the YUV data enlargedand stored in the memory 17 to the JPEG processing unit 21. The JPEGprocessing unit 21 then compresses the data to obtain JPEG data. Thefifth transfer unit 26 then transfers the JPEG data to the memory 17 forstorage and then records these data in the memory card (step A06).

Once the image recording process including the image enlarging processis executed, the process returns to step A01 to provide for the nextpick-up session.

In the above embodiment, it is assumed that 5×5 pixels including twopixels (n=2) on each side of the target pixel are required forreference, and the YUV data is formed such that blocks each containingone duplicate line (m=1) as shown in FIG. 8. However, the presentinvention is not limited to this example. The second transfer unit 18may sequentially transfer the Bayer data in units of (4n+m) (m and n arenatural numbers) lines to the Bayer/YUV converting unit 19, while thedata being shifted by 2n lines. Then, the Bayer/YUV converting unit 19generates YUV data formed of blocks each formed of (2n+m) lines based on(2n+1) pixels on each side of the target pixel. The variables m and ncan be arbitrarily set within the range of natural numbers.

For example, if n=m=2, then each block read from the memory 17 andtransferred to the Bayer/YUV converting unit 19 by the second transferunit 18 comprises 10 (=4×2+2) lines, and the starting line of each blockis shifted by four (=2×2) lines. Correspondingly, the Bayer/YUVconverting unit 19 sequentially generates the YUV data comprising 6(=2×2+2) lines including two duplicate lines.

In this case, the pixels of the Bayer data are read by the secondtransfer unit 18 in the following order. For the first block with therange of lines C1 to C10 shown in FIG. 9:

(C1, 1)→(C1, 2)→(C1, 3)→(C1, 4)→(C1, 5)→(C1, 6)→(C1, 7)→(C1, 8)→(C1,9)→(C1, 10)→(C2, 1)→(C2, 2)→ . . .

For the second block the starting line of which is shifted downward byfour lines and which has the range of lines C5 to C14:

(C, 5)→(C1, 6)→(C1, 7)→(C1, 8)→(C1, 9)→(C1, 10)→(C1, 11)→(C1, 12)→(C,13)→(C, 14)→(C2, 5)→(C2, 6)→ . . .

Accordingly, the Bayer/YUV converting unit 19 generates six lines of YUVdata from these ten lines of Bayer data. For example, the pixels of thegenerated YUV data are transferred in the following order. For the firstblock:

(C3, 3)→(C3, 4)→(C3, 5)→(C3, 6)→(C3, 7)→(C3, 8)→(C4, 3)→(C4, 4)→(C4,5)→(C4, 6)→(C4, 7)→(C5, 3)→(C5, 4)→ . . .

For the second block:

(C3, 7)→(C3, 8)→(C3, 9)→(C3, 10)→(C3, 11)→(C3, 12)→(C4, 7)→(C4, 8)→(C4,9)→(C4, 10)→(C4, 11)→(C5, 7)→(C5, 8)→ . . .

In this manner, the YUV data in the block located at a certain positionare generated so that the lower two lines of the data are overlappedwith the upper two lines of the YUV data in the next block, as shown bythe hatching in FIG. 9.

In this case, a specific process executed by the pixel number convertingunit 31 comprises referencing 4×4 pixels of the YUV data to generatepixel data to be located in the center of these pixels, byinterpolation. Since two duplicate lines are added to the YUV data blockas the fifth and sixth lines, pixel data for an interpolating linebetween the fourth line of this data block and the first line of thenext data block can be generated by using the YUV data in the third tosixth lines (four lines) and referencing an area of 4×4 pixels in eachof these lines when the fourth line of the YUV data is to be processed.

As described above, according to this embodiment, the second transferunit 18 transfers the Bayer data to allow the Bayer/YUV converting unit19 to generate the YUV data blocks each having m duplicate lines so thatthe image is not degraded when the pixel number converting unit 31executes an enlarging process for electronic zooming. Therefore, theimage is prevented from being degraded during electronic zooming withoutusing any complicated circuit configuration.

In the first embodiment, only color image data is processed. However,monochromic image data can be similarly and easily processed byexecuting a similar process while noting only Y, a luminance componentof YUV data.

Other embodiments of the digital still camera according to the presentinvention will be described. The same portions as those of the firstembodiment will be indicated in the same reference numerals and theirdetailed description will be omitted.

Second Embodiment

A digital still camera according to the second embodiment of the presentinvention will be described below with reference to the drawings.

FIG. 10 shows the configuration of a circuit in the camera, and isessentially similar to that shown in FIG. 6.

The pixel number converting unit 31 is placed immediately succeeding tothe Bayer/YUV converting unit 19. The Bayer/YUV converting unit 19supplies its outputs to the JPEG processing unit 21 and third transferunit 23 via the switches 20 and 22.

Next, the operation of this embodiment will be described.

FIG. 11 shows the contents of a process executed depending on whether ornot an electronic zooming operation is to be executed in the pick-upmode. First, it is determined whether or not the electronic zoomingfunction is turned on using the zoom key included in the key input unit29 (step B01).

If the electronic zooming function has not been turned on, the secondtransfer unit 18 reads the Bayer data in units of eight lines from thememory 17 with the upper four lines being overlapped with the lower fourlines of the upper block, as shown in FIG. 5A. The second transfer unit18 then transfers this block to the Bayer/YUV converting unit 19.

Thus, YUV data blocks sequentially generated by the Bayer/YUV convertingunit 19 on the basis of the Bayer data contain no duplicate pixels. TheYUV data blocks sequentially generated pass through the pixel numberconverting unit 31 and switch 22 to reach the third transfer unit 23.The third transfer unit 23 transfers the data blocks to the memory 17,where the data blocks are expanded and stored. The fourth transfer unit24 then transfers the data blocks to the display control unit 25. Theliquid crystal monitor of this digital still camera subsequentlydisplays and outputs the data blocks (step B03).

If it is determined at step B01 that the electronic zooming function isturned on, the second transfer unit 18 reads every block of Bayer datacomprising nine lines from the memory 17 as shown in FIG. 8. The secondtransfer unit 18 then transfers this block to the Bayer/YUV convertingunit 19.

For example, the pixels of the Bayer data are read by the secondtransfer unit 18 in the following order. For the first block with therange of lines C1 to C9 shown in FIG. 9, described above:

(C1, 1)→(C1, 2)→(C1, 3)→(C1, 4)→(C1, 5)→(C1, 6)→(C1, 7)→(C1, 8)→(C1,9)→(C2, 1)→(C2, 2)→ . . .

For the second block the starting line of which is shifted downward byfour lines and which has the range of lines C5 to C13:

(C1, 5)→(C1, 6)→(C1, 7)→(C1, 8)→(C1, 9)→(C1, 10)→(C1, 11)→(C1, 12)→(C1,13)→(C2, 5)→(C2, 6)→ . . .

Accordingly, the Bayer/YUV converting unit 19 generates five lines ofYUV data from these nine lines of Bayer data. For example, the pixels ofYUV data generated are transferred in the following order. For the firstblock:

(C3, 3)→(C3, 4)→(C3, 5)→(C3, 6)→(C3, 7)→(C4, 3)→(C4, 4)→(C4, 5)→(C4,6)→(C4, 7)→(C5, 3)→(C5, 4)→ . . .

For the second block:

(C3, 7)→(C3, 8)→(C3, 9)→(C3, 10)→(C3, 11)→(C4, 7)→(C4, 8)→(C4, 9)→(C4,10)→(C4, 11)→(C5, 7)→(C5, 8)→ . . .

In this manner, the YUV data in the block located at a certain positionis generated so that the lowermost line of the data is overlapped withthe uppermost line of the YUV data in the next block, as shown by thehatching in FIG. 8.

On the YUV data block generated so as to contain one duplicate line, thepixel number converting unit 31 executes a pixel converting processincluding an image enlarging process corresponding to the currentenlargement ratio. During this enlarging process, as specificallydescribed above in the first embodiment, the YUV data block isintentionally transferred so as to partially overlap the following datablock as described above. Consequently, when pixels lying between blocksare generated, a generated image is not degraded.

Then, the third transfer unit 23 transfers YUV data blocks sequentiallyobtained through the enlarging process and each having an increasednumber of lines (pixels), to the memory 17, where the data blocks areexpanded and stored. The fourth transfer unit 24 then transfers each YUVdata block to the display control unit 25. The liquid crystal monitor ofthis digital still camera displays and outputs a required display range(step B02).

The monitor display process is thus executed at step B02 or B03depending on whether or not the electronic zooming function is turnedon. Then, while repeatedly executing a process of determining whether ornot the shutter key included in the key input unit 29 is operated (stepB04) and if not, returning to the process starting with step B01, theapparatus waits for the shutter key to be operated.

If it is determined at step B04 that the shutter key is operated, thenit is determined whether or not the electronic zooming function isturned on (step B05).

If the electronic zooming function has not been turned on, a normalimage recording process is executed. That is, Bayer data obtainedthrough an image pick-up operation performed by the CCD 12 at a nexttiming will be transmitted to the first transfer unit 16 via the sampleand hold circuit 13, A/D converter 14, and line processing unit 15. Thefirst transfer unit 16 then transfers the Bayer data to the memory 17for storage. Then, the second transfer unit 18 reads every block of theBayer data comprising eight lines from the memory 17. Subsequently, theBayer/YUV converting unit 19 generates the YUV data blocks eachcomprising four lines and having no duplicate lines as shown in FIG. 5A.

The YUV data blocks sequentially generated by the Bayer/YUV convertingunit 19 and each comprising four lines are sequentially transferred tothe JPEG processing unit 21 via the switch 20. The JPEG processing unit21 executes a data compressing process on the YUV data in units of twoblocks each comprising eight lines to obtain JPEG data in units block(step B08).

The fifth transfer unit 26 then transfers one frame of JPEG data to thememory 17 for storage and then records these data in the memory card(step B09).

Once the normal image recording process is executed, the process returnsto step B01 to provide for the next pick-up session.

If it is determined at step B05 that the electronic zooming function isturned on when the shutter key is operated, then it is determinedwhether or not the enlargement ratio specified by operating the zoom keyis set at just 1:2 (step B06).

If the enlargement ratio is not set to 1:2, an image recording processincluding an image enlarging process is executed. The Bayer dataobtained through an image pick-up operation performed by the CCD 12 at anext timing will be transmitted to the first transfer unit 16 via thesample and hold circuit 13, A/D converter 14, and line processing unit15. The first transfer unit 16 then transfers the Bayer data to thememory 17 for storage. Then, the second transfer unit 18 reads everyblock of the Bayer data comprising nine lines from the memory 17.Subsequently, the Bayer/YUV converting unit 19 generates YUV data blockseach comprising five lines including one duplicate line as shown in FIG.8.

The pixel number converting unit 31 then enlarges the YUV data blockgenerated by the Bayer/YUV converting unit 19 and comprising five lines,so as to increase the number of pixels (lines) according to thecurrently set enlargement ratio. Then, the pixel number converting unit31 sequentially transfers the YUV data to a buffer memory (not shown)via the switch 20. The YUV data is expanded and stored in the buffermemory until storage of one frame of data is completed.

Once one frame of YUV data is stored in the buffer memory, eight linesof YUV data, which are suitable for a JPEG process, are read from thebuffer memory and transferred to the JPEG processing unit 21. The JPEGprocessing unit 21 compresses the transferred data to generate the JPEGdata (step B07).

Thereafter, the fifth transfer unit 26 transfers the JPEG data thusgenerated to the memory 17 for storage and then records the data in thememory card (step B09). Once the image recording process including theimage enlarging process is executed, the process returns to step B01 toprovide for the next pick-up session.

If it is determined at step B06 that the electronic zooming function isturned on and that the enlargement ratio is set to just 1:2, an imagerecording process including an image enlarging process is executed. TheBayer data obtained through an image pick-up operation performed by theCCD 12 at a next timing will be transmitted to the first transfer unit16 via the sample and hold circuit 13, A/D converter 14, and lineprocessing unit 15. The first transfer unit 16 then transfers the Bayerdata to the memory 17 for storage. Then, the second transfer unit 18reads every block of the Bayer data comprising nine lines from thememory 17. Subsequently, the Bayer/YUV converting unit 19 generates YUVdata blocks each comprising five lines including one duplicate line asshown in FIG. 8.

The YUV data blocks generated by the Bayer/YUV converting unit 19 andeach comprising five lines are sequentially supplied to the pixel numberconverting unit 31. The pixel number converting unit 31 then enlargesthe data so as to increase the number of pixels according to thecurrently set enlargement ratio. Then, the pixel number converting unit31 sequentially transfers the YUV data obtained and formed of blockseach comprising eight lines, to the JPEG processing unit 21.

That is, one YUV data block input to the pixel number converting unit 31comprises five lines, or four lines if the line overlapped with theadjacent block is excluded. In contrast, YUV data output by the pixelnumber converting unit 31 has their size doubled (interpolatingprocess). Accordingly, one output YUV data block comprises eight lines,which is double the size of the input YUV data block, if the lineoverlapped with the adjacent block is excluded. This size is suitablefor 8×8 pixels, a basic process unit for the JPEG process. In this case,the output data is transferred directly to the JPEG processing unit 21.The JPEG processing unit 21 then compresses the transferred data togenerate the JPEG data in block units (step B08).

The fifth transfer unit 26 transfers the JPEG data thus generated to thememory 17 for storage and then records the data in the memory card (stepB09). Once the image recording process including the image enlargingprocess is executed, the process returns to step B01 to provide for thenext pick-up session.

In this manner, the pixel number converting unit 31 is placedimmediately succeeding to the Bayer/YUV converting unit 19 so that YUVdata can be immediately compressed by the JPEG processing unit 21without expanding one frame of the YUV data on the buffer memory withregard to the data that is not enlarged or is enlarged at a particularratio. Therefore, the circuit configuration can be simplified to allowimage data to be recorded on a medium in a shorter time.

In the second embodiment, the YUV data is generated so that each blockdata contains duplicate lines. However, even if YUV data is generated soas to contain no duplicate lines as described in FIG. 5A, similareffects are produced in that YUV data enlarged at a particular ratio canbe transferred directly to the JPEG processing unit.

Third Embodiment

FIG. 12 shows the configuration of a circuit that is basically similarto that of the first and second embodiments shown in FIGS. 6 and 10.

The YUV data output by the Bayer/YUV converting unit 19 is supplied tothe pixel number converting unit 31 and switch 22. The pixel numberconverting unit 31 transmits its outputs directly and only to the JPEGprocessing unit 21 without transmitting via the switch 20.

Further, a pixel number converting unit 32 configured similarly to thepixel number converting unit 31 is placed succeeding to the switch 22.If the electronic zooming function is turned on, the pixel numberconverting unit 32 enlarges the YUV data transmitted via the switch 22and then transfers the enlarged data to the third transfer unit 23. Thethird transfer unit 23 transfers the data to the memory 17, where thedata is expanded and stored.

Now, the operation of this embodiment will be described.

FIG. 13 illustrates the specific contents of an enlarging process forelectronic zooming executed by the pixel number converting units 31 and32. For simplification, it is assumed that one YUV data block input tothe pixel number converting unit 31 (32) comprises three lines and thatthis block has three pixels arranged in the vertical direction. It isfurther assumed that one block contains one duplicate line, which isenlarged and converted into three or four pixels by the pixel numberconverting unit 31.

Reference characters P1 to P5 denote pixel data input by the Bayer/YUVconverting unit 19 and which has not been enlarged yet.

For the first block, lines corresponding to pixels P1 to P3 arranged inthe vertical direction are input; for the second block, linescorresponding to pixels P3 to P5, each of which deviates from thecorresponding pixel of the first block by two pixels, are input; for thesubsequent block, lines corresponding to pixels P5 to P7 are input, etc.In this manner, pixels are input so that a line corresponding to onepixel is duplicated between adjacent blocks.

If new pixel is generated between two vertically adjacent pixels by anenlarging process, the points at which the difference between values forthese two pixels is divided into 16 equal parts are determined by asampling operation. FIG. 13 illustrates an enlarging process at anenlarging ratio of 16/10.

If a pixel P1 is first selected and new pixels are subsequentlygenerated at the pixel intervals of 10/16, then the positions of the newpixels are as shown by the black points BP1, BP2, . . . in FIG. 13. Forexample, between points P4 and P5, pixels are generated at the positionsof the two new black points BP6 and BP7. In this case, values for thesenew pixels are each determined as shown in the following samplingoperations so as to precisely reflect the contents of a value for thecorresponding original pixel, which is closer to this new pixel on thebasis of the distance to the original pixel.BP6=(P4×14+P5×2)/16BP7=(P4×4+P5×12)/16

As shown in FIG. 13, pixel values for the four black points BP1 to BP4,including the original pixel P1, are generated from the first block,comprising the pixels P1 to P3. On the other hand, pixel values for thethree black points BP5 to BP7 are generated from the second block,comprising the pixels P3 to P5.

In this case, only the enlargement in the vertical direction isconsidered. Here, it is assumed that the original YUV data contains 100pixels in the horizontal direction and that no enlarging process isexecuted in the horizontal direction. Then, an enlarging processexecuted on the entire first block results in 400 pixels, whereas anenlarging process executed on the entire second block results in 300pixels. These pixels are transferred in the order as shown in FIG. 3.

FIG. 14 illustrates the configuration of a circuit provided in each ofthe pixel number converting units 31 and 32 to determine extractedpixels as described above, i.e. the position of a black point BPi. Inthis circuit, an adder 42 adds 21 bits of values stored in an E register41 that stores pixel intervals established after an enlarging process,to 22 bits of values stored in an A register 43. The contents of the Aregister 43 are updated, so that this sum output is stored therein.

The 22 bits of values stored in the A register 43 are output and fedback to the adder 42 as described above, while being output as extractedpixels. The A register 43 is partitioned into a B register 44, a Cregister 45, and a D register 46 as shown in FIG. 14.

In this case, the B register 44 stores the higher 8 bits of the 22 bitsto express the position of a line (a pixel in the vertical directions)in the original YUV data.

The C register 45 stores 4 bits of the 22 bits which follow the 8 bitsin the B register to express an offset position from the line positionindicated by the B register, specifically, a value for one of the pointsat which the difference between values for two pixels are divided into16 parts.

The D register 46 stores the lower 10 bits of the 22 bits and is used asa dummy register to compensate for accuracy.

For example, it is assumed that pixels are extracted as shown in FIG.13. Then, the initial value “000000(H)” is input to the A register 43,whereas the 21-bit numerical value “002800(H)” corresponding to theenlarged pixel interval “10(/16)” is set in the E register 41.

The higher 7 bits of the E register 41 form an integer part and are“000400(H)” in the case of an equal scale where no enlarging process isexecuted. Thus, the value “002800(H)” is set by:000400(H)×10/16.

That is, for the first sampling position BP1, A=“000000(H)”, B=“00(H)”,and C=“0(H)”; for the second sampling position BP2, A+E=“002800(H)”,B=“00(H)”, and C=“a(H)” (=10); for the third sampling position BP2,A+E=“005000(H)”, B=“01(H)”, and C=“4(H)”, etc. In this manner, thecontents of the A register 43 are sequentially increased by the valuestored in the E register 41, thereby allowing pixels to be extracted.

After one vertical scanning operation is completed, the scanningoperation shifts to the next pixel in the horizontal direction. Then,the contents of the A register 43 are initialized again to extractsampling positions.

Further, before the block is switched, the value stored in the Eregister 41 is added again to the contents of the A register 46 when theA register 43 is initialized. Then, this sum output is used as aninitial value for the new block.

The accuracy of the operations involved in the above-describedextraction of pixels varies depending on the size of the D register 46.In this embodiment, this accuracy is 10 bits.

The capacity of the B register 44 is set according to the maximum valueof the number of pixels (=lines) in the vertical direction, whichconstitute one block of the original YUV data. For a practical reason,about 4 to 32 pixels are used, so that this register is set to have 8bits so as to provide a sufficiently larger number of pixels.

After the pixel number converting unit 31 (32) has thus extracted pixelsand executed the corresponding image conversions as an enlargingprocess, it transfers the YUV data obtained to the buffer memory (notshown). Then, the data is sequentially expanded and stored therein untilstorage of one frame of data is completed.

Once one frame of the YUV data has been stored in the buffer memory, theYUV data in units of eight lines, which is suitable for a JPEG process,is read from the buffer memory and transferred to the JPEG processingunit 21. The JPEG processing unit 21 compresses the transferred data togenerate the JPEG data.

On the other hand, the third transfer unit 23 transfers the YUV dataenlarged by the image converting unit 32 to the memory 17 for display.When the YUV data is transferred, line number data is also transferredfor each block.

FIG. 15 indicates that the line number data “LINECNT” is transferredconcurrently with the YUV data “DATA”. FIG. 15 also shows the referenceclock “CLK”.

FIG. 15 illustrates that the line number data comprises parallel data of5-bits width on the assumption that one block can be enlarged so as tocontain up to 16 pixels (lines).

FIG. 16 indicates that to serially transfer the YUV data, the linenumber data “LINECET” is serially transferred synchronously with theenable signal “ENB” prior to the YUV data. FIG. 16 also shows thereference clock “CLK”.

In this manner, the number of pixels is increased or reduced byenlarging every block comprising a plurality of lines. If the number oflines constituting one block varies after the increase or reduction,this can be dealt with by outputting line number information to thefollowing component.

If an enlarging process for electronic zooming is executed using thesame number of pixel lines for every block, then a numeral value thatcan be set as an enlargement ratio is restricted, for example, aninteger ratio must be established between the number of pixels presentprior to enlargement and the number of pixels present after enlargement.However, this invention is not restricted by the enlargement ratiospecified for an enlarging process for electronic zooming but enablesthe electronic zooming function to be provided at various enlargementratios.

In the configuration shown in FIG. 12, the two pixel number convertingunits 31 and 32 are provided so that the pixel number converting unit 31enlarges the YUV data to be compressed by the JPE processing unit 21 andthen recorded in the memory card, while the pixel number converting unit32 enlarges the YUV data to be displayed and output by the liquidcrystal monitor or the like. This configuration is provided so as toexecute, in parallel, the processes of displaying the YUV data on themonitor and recording the YUV data in the medium, which processes theYUV data comprising different numbers of pixels, thereby allowing thedigital still camera to respond quickly without wasting time.

In the third embodiment, as in the first and second embodiments, the YUVdata is generated so that each block thereof has lines overlapped withthe next block. However, the YUV data may be generated so that blocksthereof contain no duplicate lines. In this case, different enlargementratios are used to enlarge the respective blocks, thereby allowingvarious enlargement ratios for electronic zooming to be accommodated.

In the first to third embodiments, the present invention is applied toimage data picked up by the CCD 12 in the pick-up mode. However, thepresent invention is also applicable to the image data read from amemory card in a reproduction mode.

In the first to third embodiments, the present invention is applied to adigital still camera. However, of course, the present invention isapplicable to any image processing apparatus such as a digital moviecamera, a television apparatus, or an image reproducing apparatus thathas an electronic zooming function.

Additional advantages and modifications will readily occur to thoseskilled in the art. Therefore, the invention in its broader aspects isnot limited to the specific details and representative embodiments shownand described herein. Accordingly, various modifications may be madewithout departing from the spirit or scope of the general inventiveconcept as defined by the appended claims and their equivalents.

1. A digital camera comprising: an image pick-up unit having a colorfilter including a predetermined color arrangement; a transfer unit thatsequentially transfers, in units of a first number of lines, first colorimage data output from the image pick-up unit, each unit of the firstnumber of lines including some lines included in a previous unit whichhas just been transferred; and first line number converting unit thatgenerates second color image data, in unit of a second number of lineswhich is smaller than the first number of lines, the second color imagedata including pixels which are generated based on pixels within apredetermined pixel range around each pixel in the first color imagedata transferred from the transfer unit.
 2. The digital camera accordingto claim 1, wherein the first line number converting unit generatesluminance and color difference image data, in unit of the second numberof lines.
 3. The digital camera according to claim 1, further comprisinga second line number conversion unit that carries out an enlargingprocess on the first color image data transferred from the first linenumber converting unit, thereby increasing the number of lines.
 4. Thedigital camera according to claim 1, wherein the second color image datagenerated by the first line number converting unit in each unit includessome lines included in a subsequent unit to be generated next by thefirst line number converting unit.
 5. The digital camera according toclaim 1, wherein the second color image data of the second number oflines generated by the first line number converting unit does notinclude the second number of lines to be generated next by the firstline number converting unit.
 6. The digital camera according to claim 3,further comprising a zoom instruction unit that instructs an electroniczooming operation; and wherein the second line number converting unitcarries out the enlarging process when the instruction unit instructsthe electronic zooming operation.
 7. The digital camera according toclaim 6, wherein the transfer unit sequentially transfers color imagedata in units of a third number of lines smaller than the first numberof lines when the zoom instruction unit does not instruct an electroniczooming operation, each unit of the third number of lines including somelines included in a previous unit which has just been transferred; andthe first line number converting unit generates, when the zoominstruction unit does not instruct an electronic zooming operation,color image data in units of a fourth number of lines which is smallerthan the third number of lines, the color image data including pixelswithin a predetermined pixel range in the color image data transferredfrom the transfer unit in units of the third number of lines, the camerafurther comprising a data compressing unit that compresses color imagedata transferred in units of the fourth number of lines from the firstline number converting unit when the zoom instruction unit does notinstruct an electronic zooming operation, or color image data on whichthe enlarging process has been carried out by the second line numberconverting means when the zoom instruction unit instructs an electroniczooming operation, wherein the color image data of the fourth number oflines generated by the first line number converting unit does notinclude color image data of the fourth number of lines to be generatednext by the first line number converting unit.
 8. The digital cameraaccording to claim 3, further comprising: data compressing means forcompressing color image data on which the enlarging process has beencarried out by the second line number converting unit; and a transfercontrol unit that sequentially transfers to the data compressing unit,in units of the second number of lines, color image data which has beenenlarged based on a predetermined zoom scale by the second line numberconverting unit.
 9. The digital camera according to claim 8, furthercomprising: a zoom scale specifying unit that specifies a zoom scale ofan electronic zooming operation; and a determining unit that determineswhether or not the zoom scale specified by the zoom scale specifyingunit is the predetermined zoom scale, and wherein the second line numberconverting unit enlarges, based on the zoom scale specified by the zoomscale specifying unit, the color image data sequentially transferredfrom the first line number converting unit in units of the second numberof lines, and when the determining unit determines that the zoom scaleis the predetermined zoom scale, the transfer control unit sequentiallytransfers to the data compressing unit, in units of the second number oflines, color image data which has been enlarged by the second linenumber converting unit.
 10. The digital camera according to claim 9,wherein when the determining unit determines that the zoom scale is notthe predetermined zoom scale, color image data in units of the secondnumber of lines which has been enlarged by the second line numberconverting unit is sequentially decompressed in a memory, and read, inunits of the number of lines of data compression of the data compressingunit to be transferred from the memory to the data compressing unit. 11.The digital camera according to claim 8, wherein the transfer unitsequentially transfers color image data in units of a third number oflines smaller than the first number of lines when a zoom scale in whichan enlargement by the second line number converting portion is notrequired has been specified by the zoom scale specifying unit, each unitof the third number of lines including some lines included in a previousunit which has just been transferred; and the first line numberconverting unit generates, when a zoom scale in which an enlargement bythe second line number converting portion is not required has beenspecified by the zoom scale specifying unit, color image data in unitsof a fourth number of lines which is smaller than the third number oflines, the color image data including pixels within a predeterminedpixel range in the color image data transferred from the transfer unitin units of the third number of lines, and wherein the transfer controlunit sequentially transfers to the data compressing unit color imagedata in units of the fourth number of lines which are sequentiallytransferred from the first line number converting unit when a zoom scalein which an enlargement by the second line number converting portion isnot required has been specified by the zoom scale specifying unit, andthe color image data of the fourth number of lines generated by thefirst line number converting unit does not include color image data ofthe fourth number of lines to be generated next by the first line numberconverting unit.
 12. The digital camera according to claim 3, whereinthe second line number converting unit includes means for enlargingcolor image data in units of the second number of lines which has beensequentially transferred from the first line number converting unit,each image data enlarged with a different increase number of lines. 13.The digital camera according to claim 12, wherein the second line numberconverting unit outputs information of a number of lines of color imagedata in units of the second number of lines after the image data hasbeen enlarged.
 14. An image processing method comprising: sequentiallytransferring by a transfer unit, in units of a first number of lines,first color image data output from an image pick-up unit having a colorfilter including a predetermined color arrangement, each unit of thefirst number of lines including some lines included in a previous unitwhich has just been transferred; and generating second color image databy a converting unit, in unit of a second number of lines which issmaller than the first number of lines, the second color image dataincluding pixels which are generated based on pixels within apredetermined pixel range around each pixel in the first color imagedata transferred from the transfer unit.